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[57] ABSTRACT 

A four-layer neural network is trained with data of midin- 
frared absorption by nerve and blister agent compounds (and 
simulants of this chemical group) in a standoff detection 
application. Known infrared absorption spectra by these 
analyte compounds and their computed first derivative are 
scaled and then transformed into binary or decimal arrays 
for network training by a backward-exror-propagation (BEP) 
algorithm with gradient descent paradigm. The neural net- 
work transfer function gain and learning rate are adjusted on 
occasion per training session so that a global Tni'mmum in 
final epoch convergence is attained. Three successful neural 
network filters have been built around an architecture design 
containing: (1) an input layer of 350 neurons, one neuron per 
absorption intensity spanning 700^ 1400 wavenumbers 
with resolution Av=2; (2) two hidden layers in 256- and 
128-neuron groups, respectively, providing good training 
convergence and adaptable for downloading to a configured 
group of neural IC chips; and (3) an output layer of one 
neuron per analyte— each analyte defined by a singular 
vector in the training data set Such a neural network is 
preferably implemented with a network of known micro- 
processor chips. 

19 Claims, 3 Drawing Sheets 
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NEURAL NETWORK COMPUTING SYSTEM 
FOR PATTERN RECOGNITION OF 
THERMOLUMINESCENCE SIGNATURE 
SPECTRA AND CHEMICAL DEFENSE 

The invention described herein may be manufactured, 
used or licensed by or for the government. 

FIELD OF THE INVENTION 

The present invention is related to the use of a neural 
network computing system recognizing the thermolumines- 
cence signature spectra of chemical compounds and finds 
particular utility in the recognition of nerve and blister agent 
compounds. 

DESCRIPTION OF THE PRIOR ART 

The prior art related to the present application includes the 
following references, whose disclosures are hereby incor- 
porated by reference in their entireties into this specification: 
A. H. Carrieri, 'Infrared Detection of Liquids on Terres- 
trial Surfaces by C0 2 Laser Heating AppL Opt, 
29(33), 4907^913, 1990. 
The neural network proposed is a device that will inte- 
grate to a system similar to the one disclosed in this 
publication. 

A. H. Carrieri, *Thermoluminescence Sensor for the 
Remote Detection of Chemical Agents and Their 
Simulants," U.S. Pat No. 5,241,179, assigned to the U.S.A 
as represented by the Secretary of the Navy, 31 August 1993. 

This U.S. patent discloses a Thermoluminescence (TL) 
system. The neural network of the present invention will 
perform the signal processing of mis TL sensor and similar 
systems. 

H. F. Hameka, and J. O. Jensen, 'Theoretical Prediction 
of the Infrared and Raman Spectra of O-ethyl S-2diisopropyl 
amino ethyl methyl phosphonothiolate " Int J. Quant 
Chem.y accepted for publication February 1994. 

FIG. 1 of the present application is taken from this 
publication with permission of the authors. 

M. Rrisch, J. Foresman, and A. Frisch, Gaussian 92 
User's Guide, Gaussian, Inc., © 1992. 

This is the handbook of the quantum mechanics molecular 
orbital code for predicting spectra and is used to illustrate 
how the network performance of the present invention 
compares against theoretical predictions. 

SUMMARY OF THE INVENTION 

An object of the invention is to provide a system and 
method for recognizing chemical compounds on the basis of 
their thermoluminescence signature spectra. 

To this and other objects, the invention is directed to a 
system and method in which data of midinfrared absorption 
by nerve and blister agent compounds (and simulants of this 
chemical group) are used as features for training 4-layer 
neural networks in a standoff detection application. Known 
infrared absorption spectra of these analyte compounds and 
their computed first derivative are scaled and then trans- 
formed into binary or decimal arrays for network training by 
a backward-error-propagation (BEP) algorithm with gradi- 
ent descent paradigm. The neural network transfer function 
gain and learning rate are adjusted on occasion per training 
session so that a global minimum in final epoch convergence 
is attained. 

Three successful neural network filters have been built 
around an architecture design containing: (1) an input layer 
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of 350 neurons, one neuron per absorption intensity span- 
ning 700^v^l400 wavenumbers with resolution Av=2; (2) 
two hidden layers in 256- and 128-neuron groups, 
respectively, providing good training convergence and 
5 adaptable for down-loading to a configured group of neural 
IC chips; and (3) an output layer of one neuron per analyte — 
each analyte defined by a singular vector in the training data 
set 

A weight matrix per filter yielding best performance by 

10 software simulation is transferred to a neural network which 
in a preferred embodiment includes eight interconnected 
INTEL 80170NX Electronically Trainable Analog Neural 
Network (ETANN) chips, housed arid connected on a 
mother circuit board, for real-time testing of external 

15 streams of data. These test data include infrared tr ainin g 
spectra with added noise, spectral predictions by theoretical 
model, or preprocessed spectra from a sensor unit. Detection 
features in data forwarded to the network filters are imprints 
of molecular vibrational modes by nine analyte compounds. 

20 Preprocessing of raw spectra involves digitization of inter- 
ferograms and fast Fourier transformation, spectrum 
subtraction, spectrum normalization and baseline correction, 
electronic noise filtration, and polarity adjustment of 
enhanced emission bands brought on by the sensor's irra- 

25 diating beam source. 

These neural network pattern recognition systems can 
achieve in situ spectral discrimination of organic liquid 
layers on the ground and their evaporated vapor clouds when 

3Q integrated into a mobile mermoluminescence field sensor. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Hie preferred embodiment of the invention will now be 
described in detail with reference to the drawings, in which: 
35 FIG. 1 shows a scaled infrared absorption spectrum of the 
liquid chemical nerve agent Ethyl S-(2- 
Diisopropylaminoethyl) Methylphosphonothiolate (VX); 

FIG. 2 shows a first-derivative scaled infrared absorption 
spectrum of the liquid chemical nerve agent Ethyl S-(2- 
40 Diisopropylaminoethyl) Methylphosphonothiolate (VX)of 
FIG. 1; 

FIGS. 3A-3F show neural-network failure analyses; 
FIG. 4 shows a diagram of hardware used in the present 
M , invention; and 

45 

FIG. 5 shows a training procedure used in the present 
invention. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENT 

50 

The preferred embodiment of the invention is directed to 
a Neural Network Computing (NNC) system for detecting 
surface chemical contamination by mamoluminescence. 
Features for the radiometric detection of the contaminant 

55 compound (analyte), or group of compounds, are absorption 
and emission moieties carried in a thermal energy release by 
the surface of the compound vis-a-vis stimulation by an 
irradiating beam source. Conversion of the incident beam 
single energy (energy efficiently absorbed into the material 

60 volume of the compound, and usually selected outside the 
detection band of the measuring instrument) to a broad 
thermal release over the surface boundary of the compound 
under irradiation is called mermoluminescence. 
The irradiated zone builds in temperature (T^ resulting 

65 in a shift in graybody emissions liberated by the material 
toward higher infrared (IR) energies. When this Planck 
distribution spectrum shifts at maximum rate, i.e., when 
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Vsurf) Passes through 0 (the dot symbol denotes differen- stances are encountered in recognizing patterns of thermal 

nation with respect to beam irradiation time), the surface is activity in organics; for example, the preferred embodiment 

in its most thermally dynamic state while emissiviry of the 0 f the present invention is concerned with absorption and 

infrared-actiye analyte and background medium are at their emission ^ from h vdd ^ organophosphonate 

5 contaminants brought out by a controlled beam stimulation 
fundamental vibrational modes, within this graybody frrT _ .r u XT 7 . _ ^ 
spectrum, will show their band structure prominently by a of ^ T° accomplish a Neural Network Computing (NNQ 
simple spectrum subtraction about an irradiation period system far this TL standoff detection problem, the preferred 
where M, ' is maximum, being the peak value if of the embodiment uses software called DynaMind (a product of 
thermal graybody emissions. 10 NeuroDynamX, Inc) and hardware with accessories for 
A functional and practical field sensor built around this configuring and testing eight Intel 80170NX Electronically 
principle will include a system whose optics, electronics, Trainable Neural Network (ETANN) chips (a product of 
algorithms, and neural network pattern matching computa- Intel, Inc). However, those skilled in the art who have 
tions will do a full range of TL detection tasks as stated reviewed this specification will readily appreciate that the 
Electronic and mathematical data handling operations nec- is present invention is not limited to any particular brand of 
essary to determine an in situ presence or absence of various software or hardware. The software and hardware products 
analyte compounds include: (1) A/D data acquisitions from named are compatible, so thatNNCs developed in software 
a Michelson interferometer for generating a plurality of simulation are easily downloaded to a configured and con- 
temporal radiance patterns by IR interference ^ nected hardware board 

(interferograms), (2) fast Fourier transformation (FFT) of A brief explanation of some terminology and function is 
the interf erograms for frequency (energy) deconvolution of in order before these networks are disclosed. Neural net- 
the collected radiance, (3) spectrum subtraction, baseline works are neurons or processing elements (PEs) grouped in 
correction, and positive polarity adjustment of bands for input, hidden, and output layers that communicate in parallel 
resolving the contaminant's molecular vibrational modes, 25 via full interconnections of PEs between layers. The 
(4) spectrum z-scaling for normalization of IR spectra, (5) strengths of interconnections are called weights. In t rainin g 
noise filtration, (6) neural network computing for pattern sessions of a network, learning is composed by a training 
recognition of the analyte' s vibrational spectrum, (7) map- algorithm and paradigm, causing pattern updated adjust- 
ping coordinates for contaminant locations through a Global ments in weight strengths. The training algorithm used here 
Positioning System, and a variety of systems-related data 30 is called Backward-Error Propagation (BEP), a method of 
routing and automation operations with safety checks and error analysis where perturbations put on the weights are 
feedback functions. The present invention is directed to the distributed in a manner to reduce overall network epoch 
neural network computing component of such a system. error. All weights of the network are dimensions in the 
The inventors will describe the preferred embodiment of 35 domain of the BEP transform (weight space). The learning 
the invention in the following manner. First, the inventors paradigm (convergence to a correct result) uses a gradient 
will briefly review neural network computing and design descent method for seeking global rmnima in this weight- 
then build three simplistic spectral pattera-rnatching filters dimensional space. 

in binary absorption, decimal absorption, and decimal Each of the first hidden layer PEs sums the product of 
derivative-absorption training data formats. Subsequently, each preceding input layer PE signal and its respective 
network training procedures to recognize a group of nine weight (plus bias weight, if present) and passes this value to 
analyte compounds are presented, followed by test proce- a transfer function (TF) that scales all summed inputs to 
dures using noisy training data, theoretical predictions of yield new signals conducted to a neuron in the succeeding 
spectra, and sensor data. The effects of noise on network 45 bidden layer or output layer (feed-forward topology). The 
pattern-matching performance will be examined, and a TF used here is of a hyperbolic tangent form (tanh) with 
down-loading procedure of trained network architecture and limits of ±1 for scaling all incoming PE signals , within these 
weight matrix filter to semiconductor hardware for full limits, and an adjustable transfer function gain for control- 
network integration into a mermoluminescence (TL) sensor ling the TF rate of change, also between these limits; both 
data acquisition system will be set forth. Finally, the inven- 50 parameters control conduction of PE output (firing of 
ton will suggest improvements in quantum mechanics spec- neuron). This tanh function is useful when training sets of 
tral computation methods that should improve the precision nonuniformly distributed data with densities away from the 
of predicting vibrational energy states in these analyte set mean value, as is typical of IR spectra of organic 
molecules (fed through these neural networks), discuss an 55 compounds (sometimes referred to as the fingerprint energy 
alternate network paradigm and hardware support for pos- band). 

sible improved confidence of detection, and finalfy, summa- The network topology is disorganized before training; 
nze the goals of the invention. neural pathways are randomized with no associative learn- 
Neural network systems designed to interpret infrared ing. Training data consist of nine sets of standard absorption 
spectra are successful and usefull laboratory devices. Cer- 60 spectra together with a singular vector representation per 
tain advantages of these systems are realized over statistical analyte called the target output The goal of the network is 
model linear regression approaches of spectral discrimina- to associate each class of vibrational bands (uncommon 
ton: Neural networks use nonlinear transform functions to feature of each molecular species) to its defined vector 
map nonlinear variables, they can operate on non-normal 55 representation by the BEP mapping. Learning by the net- 
distributions of data, and input data need not be linearly work is complete when error in this transformation is null, 
independent (orthogonal input vector space). Such circum- i.e. zero, the network converges yielding a global minimum 
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on the topological weight surface via gradient descent In 
gradient descent, this topological surface in weight space is 
reformed, gradually in training time (weight updates), as 
network mis error per epoch decreases 

( £ ~ 1/2 Wetworki - Target;? j 

with respect to the weight matrix 

The proportionality constant a is called the network learning 
rate. 

This prototype TL sensor uses a scanning Michelson 
interferometer for generating a plurality of temporal voltage 
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The molecular structure of nerve agent Ethyl S-(2- 
Diisopropylaminoethyl) Methylphosphonothiolate (VX), 
includes bond angles and lengths that were computed by a 
geometry optimization option in a quantum mechanics pro- 

5 gram called Gaussian 92. This compound is the first of seven 
analyte compounds whose experimentally measured vibra- 
tional spectrum in part form the training basis set of these 
networks. Table 1 at the end of this paragraph lists energies 

10 of the strongest vibrational transitions and mode assign- 
ments in VX and six other analyte compounds of the 
network training data set The strongest vibrational modes in 
each molecular species represent singular classes of detec- 
tion features. (The dipole moment derivative quantity 

15 between a bonded atom pair in the molecule is responsible 
for the IR band and its strength of transition.) 



TABLE 1 



Analyte 
Nomenclature (TJ) #) 


Formula 


Band 


Vibrational 
Energy 
(cm- 1 ) 


Sealed 
Absorption 
Intensity 


Mode 
Assignment 


VX(0) 


CH 3 CH 2 0P(O>— 


1 


894 


0.2498 


P2— CoHj rode 


etb^2dlisopropylamin0ethyl) 




2 


956 


0.4380 


C3U2 bend 


Methyl-phosphonothiolate 




3 


1036 


13941 


P2-05-C10 stretch • 




4 


1230 


0.7388 


C3Hj bend 


DIMP(2) 


OP(OCH(CH 3 ) 2 ) 2 CH3 


1 


984 


0.8883 


C— O— P— O— C stretch 


diisoprupyhnetbyl- 


2 


1012 


0.6748 


C—O—P—O—C stretch 


pbasphonatc 




3 


1110 


0.2483 


0 — C 3 stretch 




4 


1244 


0.6149 


P=0 stretch 


MMP(3) 


OPfOCH^CHa 


1 


820 


03010 


P— C stretch 


dimcthylmcthyl-phos0iOTiatc 




2 


1032 


1.2025 


C—O—P—O—C stretch 




3 


1058 


0.6389 


C—O—P—O—C stretch 






4 


1246 


0.5877 


P=0 stretch 


GA(4>Tabua 


CjHsOPCOKC^CHj^ 


1 


1006 


0.8612 


P — 0 — C stretch 


ethyl n-dimethyl- 




2 


1030 


0.7716 


C— N— Cbend 


phosphoramkb- 




3 


1268 


0.7235 


P — 0 stretch 


cyanidate 




4 


1320 


0.4208 


N — C stretch 


GB(5)-Sarin 


CH^OXFOOCiC^H,) 


1 


838 


03174 


P—F stretch 


iospropylmethyl- 
phosphonofiuoridate 




2 


1014 


1.1719 


P — 0 stretch 




3 


1278 


0.6790 


P=0 stretch 






4 


1320 


03486 


C— H bend 


GD(6)-Soman 


CH^^OCCH,)- 


1 


986 


0.4235 


^ rock 


methyl 1 -methyl- 




2 


1000 


0.4557 


P — O stretch 


2-dimethylpropyl- 




3 


1018 


1.1152 


P — O stretch 


pho6phonofluoridate 




4 


1284 


0.7892 


P=0 stretch 


SF96(8) 


[-^(CI^O-L 


1 


800 


0.7581 


Si— CHj rock 


polydimethybUoxane 




2 


1022 


0.5513 


St— O— Si stretch 




3 


1094 


0.5388 


St— O— Si stretch 






4 


1262 


0.6088 


St— CH 3 sym stretch 



waveforms called interferograms (time-dependent intensity 50 
patterns of constructive and destructive interference in col- 
lected IR waves). The interferograms data rate is dictated by 
a period of constant velocity movement in the interferom- 
eter's oscillating mirror. Measurement of the interferograms 
is done by amplifying, digitizing, and storing signal output 
from a cooled MCT detector chip located in the optical path 55 
after the interferometer and in the focus of the incident TL 
radiance. This interferogram is transformed into IR fre- 
quency amplitudes by an on-board fast Fourier transform 
(FFT) algorithm (part of the sensor array-processor data 
acquisition and preparation unit). Contiguous sets of spectra 60 
are subtracted in a period of beam irradiation causing a 
developing thermal gradient in the sample to maximize. 
These difference FFT intensities are preprocessed men con- 
figured into binary or direct analog arrays of data covering 
700 £v=2 1400 wavenumbers (wn's) in 2 wn resolution, 65 
buffered, the n sent to the neural network input layer (350 
simultaneous intensities of spectral data). 



Training and testing data representations in this NNC 
system take on three separate structures. For example, the 
binary structure from the spectrum of VX shown with the 
solid line in FIG. 1 was produced from experimental mea- 
surement of neat liquid VX by a Nicolet 10-DX FITR 
instrument 

FIG. 1 shows a scaled infrared absorption spectrum of the 
liquid chemical nerve agent Ethyl-(2- 
Diisopropylaminoethyl) Methylphosphonothiolate (VX). 
The dashed curve is a theoretical prediction. The solid curve 
is an experimental measurement where numbers above the 
peaks identify four key vibrational modes in the VX mol- 
ecule. The horizontal line A'=0.2498 corresponds to the 4th 
strongest vibrational intensity; a binary divider of this spec- 
trum: binary+1 for A ^0.2498 and binary -1 for A<0.2498. 
Training data are of a 25x14 binary matrix form: 
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1111111 





































The array bits arc separated by 2 can" 1 , read left-to-right by 
row (element [1,1] is binary A 1 (702) . . . element [25,14]is 
binary A' (1400)). The right-most column is a vector asso- 
ciating this map to VX. This binary matrix makes up one 
training set exemplar. A variety of mathematical operations 
arc performed on raw data (stored laboratory spectra) to 
produce this; exemplar. They include scaling by the set 
expression 

where brackets denote mean value, the subscript denotes IR 
absorption frequency (PE number 0 ... 349 of the network 
input layer), and s is the standard deviation. This will 
standardize the intensity scale of spectra read in from 
different spectrometer systems, i.e., it allows a common 
ordinate axis for the network training data set and incoming 
sensor data sets. It also provides a range of spectral ampli- 
tudes compatible to the tanh transfer function limits set in 
the network semiconductor hardware. The next preprocess- 
ing operation applies a noise reduction filter the scaled 
spectra if extraneous signal is present Savitzky-Golay type 
digital smoothing polynomial filters achieve limited success 
for this operation. The scaled and smoothed spectrum is next 
prepared for binary encoding. Binary values are grouped by 
first locating the forth- strongest absorption peak intensity in 
each spectrum. A +(-)l bit is assigned to each datum of the 
scaled spectrum whose intensity lies at or above (below) the 
4th -peak intensity divisor. The reason for locating the 
fourth-peak intensity is that, within the bandwidth of the TL 
sensor, the quantity of strong vibrational states in these 
analyte molecules do not usually exceed four. The bottom of 
FIG. 1 shows the resultant 25x14 binary field with a singular 
9-dimensional vector (one dimension per training set exem- 
plar in the training data block) in the right-most column to 
associate this binary field to the chemical agent VX. 

When detecting multiple contaminants, the network 
binary filter must recognize groups of spectrum bands per 
analyte in constant intensity proportion. A preprocessing 
algorithm that cascades binary data fields is used for this 
task. The procedure first involves submission and forward- 
ing of the fourth-peak binary field (FIG. 1) to the trained 



25 network. The network is next checked for a positive iden- 
tification. If that check is negative, the algorithm continues 
on locating a fifth peak and, if present, generates a new 
binary field where the binary divider correlates to this scaled 
fifth-absorption bandhead. The algorithm continues seeking 

30 peaks and submitting binary files to the network until an 
offset peak intensity is reached — an intensity set above the 
noise level of the spectrometer instrument 

The second data structure uses the same 25x14 matrix 
format as with the binary field except here the absorption 

35 spectra, in a string of decimal data points, are written 
directly into the matrix after D/A conversion of the preprc— 
cessed FFT. This decimal method is more direct, less time 
consuming, and slightly more precise than that of the binary 
method of training and feed-forwarding to the NNC. 

40 However, hardware is now more complex and the data 
acquisition system more bulky and difficult to calibrate since 
the spectrometer electronics (a digital data acquiring 
instrument) must now include multiplexed A/D converters 
with 350 channels, once channel per input PE. 

45 The last data structure designed, built, and evaluated for 
training by this same network architecture is identical to the 
above absorption structure, except a differentiating opera- 
tion is added after FFT subtraction and preprocessing opera- 
tions. The goal was to increase network specificity of 

so detection by applying the energy derivative 

3A 

55 operator just before submission, of the spectrum to the 
network, especially where overlapping band structure is 
present The inventors' experience in testing neural net- 
works with derivative-based spectra suggests that prc- 
filtering and smoothing operations are necessary and require 

60 special handling in preprocessing, since noise by the instru- 
ment can cause substantial error peaks in A' (about abrupt 
adjacent data points). 

FIG. 2 shows the smoothed derivative absorption training 
spectrum of FIG. 1, with its decimal training data set 

65 representation. 

These network training data are of a 25x14 decimal 
matrix form shown below. The array numerics are spearted 
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by 2 cm" 1 , read left-to-right by row (element [1,1] is decimal network net file. Weights are initialized before training the 
A' (702) . . . element [25,14] is decimal A* (1400)). The network, and a filter function is used during training sessions 
right-most column is a vector associating this map to VX. to accept only those weight values between ±2.5, the allow- 



0.068 


0.035 


-0.000 


-0.062 


-0.060 


-0.085 


-0.102 


-0.118 


-0.123 


-0.120 


-0.115 


-0.097 


-0.082 


-0.067 


-0.047 


-O.050 


-0.065 


-O.087 


-0.074 


-0.037 


0.055 


0.153 


0.236 


0.258 


0.221 


0165 


0.112 


0.067 


0.037 


0.009 


-0.040 


-0:055 


-0.090 


-0.135 


-0.187 


-0.225 


-0.192 


-0.057 


0.105 


0.218 


0,246 


0.210 


0.150 


0.100 


0.063 


0.039 


0.032 


0.029 


0.027 


0.022 


0.015 


0.007 


-0.000 


0.002 


0.007 


0.017 


0.014 


0.015 


0.012 


0.005 


0.007 


0.000 


0.002 


-0.000 


-0.000 


-0.005 


-0.002 


-aooo 


0.002 


-0.000 


0.000 


-0.002 


-0.009 


-O.014 


-0.017 


-0.020 


-0.020 


-0.027 


-0.025 


-0.045 


-0.069 


-0.100 


-0.143 


-0.198 


-0.263 


-0.340 


-0.386 


-0368 


-a 220 


-O.014 


0.127 


0.107 


0.017 


-0.120 


-0.205 


-a 138 


0.067 


0.290 


0374 


0354 


0.300 


-O.230 


0.309 


0.098 


0.058 


0.034 


0.014 


-0.002 


-0.027 


-0.052 


-0.082 


-0.107 


-0.127 


-0.137 


-0 137 


-0.128 


-0.117 


-O.109 


-O.120 


-0.135 


-0.163 


-0.200 


-0.242 


-0.286 


-0310 


-0.285 


-0.180 


0.010 


0216 


0.782 


0.436 


0.414 


0373 


0.308 


0.236 


0.180 


0.135 


0.102 


0.070 


0.049 


0j034 


osm 


0.014 


0.009 


-0.002 


-0.009 


-O.015 


-0.024 


-0.029 


-0.039 


-0.052 


-0.065 


-0.082 


-0.107 


-0.140 


-0.170 


-a 198 


-0.227 


-0.335 


-0.443 


-0366 


-0.697 


-0.833 


-0.973 


-1.084 


-1.098 


-0.827 


0.181 


1.056 


1.104 


0.945 


0.808 


0.640 


0.494 


0388 


0.306 


0.233 


0.172 


0.143 


0.137 


a 135 


0.118 


0.185 


0.080 


0.083 


0.088 


0.083 


0.072 


0.055 


0.039 


0.020 


0.007 


-0.007 


-0.025 


-0,047 


-0.074 


-0.097 


-0.168 


-0.025 


0.012 


0.019 


-O.004 


-0.030 


-0.042 


-0.069 


-0.094 


-0.030 


0.080 


0.113 


0.067 


-0.019 


-0.062 


-0.057 


0.020 


0.107 


0.128 


0.102 


0.065 


0.030 


0.002 


-0.027 


-0.057 


-0.099 


-0.147 


-0.250 


-0330 


-0.386 


-0341 


-0.102 


0.177 


0.280 


0.245 


0.215 


0.225 


0.225 


0.196 


0.281 


0.090 


0.055 


0.029 


0.007 


-0.012 


-0.027 


-0.044 


-0.067 


-0.099 


-0.138 


-0.190 


-0.238 


-0.261 


-0.256 


-0.220 


-0.217 


-0.246 


-0.258 


-0.192 


-0.087 


-0.060 


-0.120 


-0.223 


-0328 


-0.399 


-0.414 


-0384 


-0.019 


0.153 


0.233 


0.230 


0.251 


0318 


0.434 


0.522 


0.532 


0.466 


0.379 


0.301 


0.210 


0.163 


0.127 


0.097 


0.075 


0.057 


O047 


0.037 


0.027 


0.020 


0.012 


0.005 


-0.004 


-O.015 


-0.029 


-0.065 


-0.104 


-0,170 


-0.271 


-0.409 


-a 544 


-0396 


0.032 


0.251 


0306 


0314 


0.286 


0.243 


0.158 


0.098 


0.058 


0.032 


-0.004 


-0.032 


-0,027 


0.009 


0.053 


0.055 


0.030 


0.030 


0.042 


0.034 


0.019 


0.012 


-0.004 


-aoi9 


-0.039 


-0.067 


-0.084 


0,097 


-0.135 


-0.198 


-0.265 


-0.293 


-0.172 


0.123 


0.305 


0.462 


0.077 


-0.049 


-0.137 


-0.170 


-0.175 


-0.185 


-0.102 


0.055 


0.093 


0.024 


0.019 


0.112 


0.193 


0.167 


0.112 


0.062 



Training each network to its data structure is done by use 
of the DynaMind software product, BEP training algorithm 
option with gradient descent paradigm. Each preprocessed 
data structure is loaded on computer file with a suffix io; that 
file contains a header block with quantity input and output 
layer PEs and comment information. Input, hidden, and 
output layer PEs are keyed into the program per network file 
name and description. The input layer is a 350 dimensional 
space, and quantity of PEs in the output layer is nine, or one 
neuron per analyte contained in the training data set. 

Network hidden layers and PEs per layer were determined 
through trial-and-error training experiments in software 
simulation. Starting with a single hidden layer of 175 PEs, 
rates of network convergence and final epoch error were 
checked by observing the rms error instrument brought on 
screen during training sessions. Since the weight matrices 
developed after network training are basic spectral look-up 
tables, and the input space is one running variable in energy, 
rms error was chosen as the key network performance 
parameter of training. An increase of hidden layer PEs from 
175 to 550 (experiments were done in steps of about 100 
PEs) produced the best single hidden layer network perfor- 
mance result For IC hardware compliance reasons, this was 
changed to a modularized 2-hidden layer architecture of 256 
and 128 successive groups of PEs. (Reducing network 
complexity by pruning hidden layer PEs that do not con- 
tribute to performance, thereby increasing network 
throughput, is not a current option in the version of Dyna- 
Mind code used.) 

Intel's 80170NX ETANN chip architecture, onto where 
the network weight matrix is down-loaded, is restricted to a 
maximum of 64 PEs per hidden and output layers and 128 
PEs input layer mode, and the quantity of chips on one bus 
(EMB printed circuit board) is limited to eight It will be 
explained below how the above four-layer network archi- 
tecture (one input, two hidden, one output) meets the hard- 
ware constraint when the trained weight matrix is modular- 
ized and implemented onto eight interconnected Intel 
80170NX IC chips. The network tanh TF limits, learning 
rate, and gain parameters are next programmed to create a 



able range for the ETANN chips. The steps per training 
epoch are fixed at nine, or one sweep of neurons in the 
network output layer. 

The DynaMind executable now has all necessary infor- 
mation for start-up of a training session, or a resume of 
training. Training sessions are periodically interrupted to 
change learning rate and TF gain parameters, then re- started, 
to mimrnize epoch error when producing these fine-tuned 
weight matrices (filters) by absorption-binary, absorption- 
decimal, and derivau^e-absorpu^n-o^cimal structured data 
formats. Epoch errors below 10~ 15 were attained in final 
training convergence by all three network training structures 
using this method of parameter adjustment for seeking, 
centering, and settling to the global nunimum. 

On completion of training, DynaMind writes the four- 
layer network weight matrix result into a network.net file; 
each nine-analyte pattern matching filter occupies about 1 
MB of computer RAM memory per data structure and 
network architecture previously discussed. Pattern recogni- 
tion from a stream of external spectral data, in forms of 
preprocessed sensor, theoretical prediction, or manipulated 
training data sets is started by first loading the network.net 
and dataio files into CPU memory, linkage, then CPU 
access to the program executable via a mouse click-on 
maneuver of a network file called net set The network of the 
fourth-peak binary structure was tested first, and the results 
are shown in Table 2 below. 
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Binary Data Set Repiese 


station 




Incorrect 
Identi- 
fication 


Training 
Spectrum 


Testing 
Spectrum 


Noise 
Added 
(%) 


Correct 
Identification 
Agent (% true) 


Agent 
(% 

False) 


8 

A" 


Ac*, Ai* t A a * 
A3*,A4* As* 
Ar 8 , At* 


15 


0(98.7), 1(98.7), 
2(98.7) 

3(98.7), 4(98.7), 
5(98.7) 


None 
None 
None 




Afl 1 *, Ag B , Aj® 

Aj » A4 , Ag 


23 


6(98.7), 7(98.7), 
8(98.7) 

0(98.7), 1(98.7), 
2(98.7) 

3(98.7), 4(98,7), 
5(98.7) 

6(975), 7(98.7), 
8(98.7) 


Nose 

None 

8 -> 
5(89.4) 




To» 

T3* 

T 4 * 
T5* 
T«* 


0 


No ID 
No ID 
No ID 
No ID 

6(90.0) 


None 

None 

None 

6(98.4) 

2(98.7) 


i=0 


Si* 


0 


2(98.7) 
3(98.7) 
8(98.7) 


None 
None 
None 
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The first test was feed-forwarding the training exemplars 
TABLE 2 A B back through the binary BEP-trained network The 

network performed one of nine detections with an expected 
accuracy of 100% true predictions and 0% false identifica- 
5 tions. The percentage value is an inner product computation 
of the analog nine-component response by the network 
output layer and the training set epoch vector for each of the 
nine contaminant compounds. (In feed-forwarding, the tar- 
get output vector is programmed as an unknown; the net- 
work must select any of the nine analyte(s) possessing 
spectral content in the input exemplar, or reject all nine 
analyte compounds on completion of its pattern search.) 
The next set of network performance tests was carried out 
15 with binary training data altered in a way to simulate noisy 
spectral data generated by an external sensor source. In this 
scheme, a percentage of negative (-1) bits in each exemplar 
of the data block representing all scaled analyte spectra was 
randomly reversed, then feed-forwarded it through the net- 
20 work. Partem matching performance was excellent; this 
network weight matrix successfully filtered about 18% 
binary noise yielding accurate identifications of all nine 
contaminants. By adding 20% noise to the training data and 
feed-forwarding, the binary filter began breakdown by 
25 falsely detecting agent 5 (GB) in the agent simulant 8 (SF96) 
exemplar to 89.4%. The clean and noisy binary-encoded 
spectra of SF96 (exemplar 8) are graphed in FIGS. 3A and 
3B 

Binary Divider is the intensity of the 4th strongest peak per scaled absorption 

spectrum. 30 To achieve the results shown in FIGS. 3A-3F, the inven- 

Input layer neurons = 350, 1st hidden layer neurons = 256 and 2nd hidden t gradually increased noise in the training data set until 
layer neurons = 128, output layer neurons « 9, epoch error < 1CT 15 , A B « & , * „ uouuu B u»ux dqu uum 

binary IR absorption spectrum (Nicolet 10-DX spectrometer measurement), «he network failed to identify the analyte, Until the network 

i = (0, l , 2, 3, 4, 5, 6, 7, 8) s (vx, Vx, DMP, DMMP, gagb, gd, gf, SF96), made incorrect identifications, or both failure modes were 

W^^SS^tS^S present Column A (FIGS. 3A, 3C and 3E) is the analyte 

0.010/2.00, sigmoid limits « ±1, and weight filter = ±2.5 35 compound training data set plotted in: 4th-peak binary (top, 

Given are network neurons per input-hidden-output SF96 s8 >> scaled dQcim21 *«*P*» (middle, GtaQ, and 

layers, converged epoch error, and tanh transfer function scaled dedmal derivative-absorption (bottom, GA-4) for- 

parameters including gain and learning rate. The summed ^ < Scc TaD - *) Column B (FIGS. 3B, 3D and 3F) shows 

A-symbol suggests a binary format of 9 scaled infrared 40 ^ c noisc threshold for first network failure: 20% in 

absorption spectra. A horizontal line across each spectrum B(SF96), 27% in B(GD) and 19% in B(GA). In B(SF96), the 

corresponds to its 4th-most intense scaled absorption 4th-peak based binary filter identified two compounds: cor- 

intensity— it is the binary divider of the spectrum (FIG. 1). red identification of SF96 to 98.7% certainty, and an incor- 

These nine binary fields, with noise added, were submitted rect identification of agent GB to 89.4% certainty. In B(GD), 

to the network filter as test data sets. Noise is defined as 45 the absorption based filter also identified two compounds: a 

reversing a percentage of negative bits in the binary field of correct identification of agent GD to 98.7% certainty, and an 

each analyte. At 20% noise error, the network begins pro- incorrect identification of agent GF to 90.3% certainty. In 

ducing false identifications, and the deterioration is chaotic B(GA) the derivative-absorption based filter failed to iden- 

beyond this value. Testing the trained network is also tify any of the 9 analyte compounds. (See Tabs. 2r-3.) Noise 

performed with scaled, binary-coded, theoretical spectral above the tabulated values cause rapid deterioration in 

predictions (T*), and spectra measured by a TL sensor (S*). ability of these network filters to pattern-match one of nine 

Correct and incorrect identification percentages are com* compounds correctly. 

puted by the inner product of training vector and network Other absorption test data submitted to the network binary 

vector read directly from each neuron in the network output 55 filter came from ab initio closed-shell molecular orbital 

layer (analog outputs). (MO) calculations on each analyte molecule (T*, T*) by a 

In Table 2, the summed A- symbol suggests a training set popular program commonly used in quantum chemistry 

data block of nine compounds (with vector definitions) called Gaussian 92. A Hartree-Fock(HF) approximation was 

comprising the training io file with header information used with the 3-2 1G* wavefunction basis set option for 

(comments on problem definition and network purpose, and 60 producing these spectral predictions. The network is put to 

quantity input and output PEs). Molecular descriptions of use as a system for checking validity of these quantum 

the analyte compounds and their training properties per data mechanics calculations against controlled experimental 

block are given in Table 1. Each block fills the computing measurements (builders of the network filters). All spectral 

capacity of a 32MB Random Access Memory (RAM), 50 65 predictions by Gaussian were produced without knowledge 

MHz, 80486 PC machine executing DynaMind under the of experimental data. Five of nine molecules, whose experi- 

MS-DOS operating system. Version 5.0. mental spectra arc ingrained in the binary filter, were ini- 
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tialized in Gaussian 92 format aad successfully executed by system with improved twowavenumber interferometer and 

the time of this writing: VX, DMMP, GA, GB, and GD, microwave TL sdmulator.) 

resenting T 0 *,T 3 * T/, T S B ^ and V; respectively. The ^ ^ tQ ^ QetWQrks wereproduoed 

VX calculation is supenmposed on its experimental coun- ^ . . . , m , . ^ . , , 

tcrpart in FIG 1 5 from the inventors work on TL detection of liquid layers 

Table 2 indicates an inability of the binary network filter Md evaporated vapors of organics DMP (S A DMMP 

to correctly pattern-match theory and experiment in all (S2*) and SF96 (Sg*) wetting various terrains. The results in 

compounds except GD. Moreover, the GD correct detection Table 2 show (hat the binary neural filter detected one of 

event is shared with a false identification of G-agent simu- nine analyte compounds accurately with no fake detections, 

lant DIMP. These results suggest that accuracy of the ^ ^deed a remarkable achievement given the senor 

calculation of vibrational energies and intensity of these ^ m ^ ^ ^ m ^ lower m baDd 

energy transitions in molecules the size of VX (and higher r , , t _ rf . . 

molecular weight) is critical. of fte measunn 8 mstrumfint Currently, the inventors are 

The third and most important test of these binary network 15 restructuring the binary (and decimal) filters for any of N 

filters comes from forwarding of data generated by a pro- analyte detections. The training data will require known 

totype TL sensor (S*). These raw sensor data were derived absorption cross-sections for each major vibrational reso- 

frora the difference of two slightly shifted graybody spectra nance in all analyte compounds, viz, quantitative data are 

measured during irradiation by a laser source beam emitting required for creating exemplars of mixed compounds. The 

at 829 cm' 1 . It was necessary to block all scattered laser 20 ^ to block ^ now ^st of 2"-l exemplars, N 

hght from entering me mterferometer by luting a cutoff ^ ayaflablc ff ^ m 
CaF wmdow in the entrance aperture just before the inter- 

ferometer beamspUtter. The CaF crystal did block the laser Table 3 below Ms Performance results on the same 

scattering energy, but it also caused attenuation of TL network architecture used to build the binary filter, except 

collected between 700 to about 900 cm' 1 . (Hie CaF band- 25 now it is trained against pre-processed IR decimal absorp- 

pass cutoff was not as sharp as desired) The other anomaly tion (page 19 of Table, A**) and first-derivative absorption 

of these sensor spectra are their 4 cnrtesolution — twice (page 20 of Table, A°) analyte spectra. The inventors added 

less resolved than the spectra used to create the network a rule in the decision columns: reject identification by the 

binary filter. (Both of these problems are remedied in a new filters, correct or incorrect, below a 90% confidence limit 



TABLE 3 


Decimal Data Set Representation 






Training 
Spectrum 


Testing 
Spectrum 


Added 
Noise 
(ft) 


Correct ID 
>90% Agent 
(ft True) 


Incorrect ID 
>90% 

Agent (% False) 


I AP 

F=0 


Ai^APtAs 0 
M D ,AP f At D 


25 


0(96.7), 1<98.7),2<98.7) 
3(98.7), 4(98.7), 5(98.7) 
6(98.7), 7(98.7), 8(98.7) 


None 
None 
None 




A 4 D f A T D ,A t D 


27 


0(98.7), 1(98.7), 2(98.7) 
3(98.7), 4(98.7), 5(98.7) 
6(98.7), 7(98.7), 8(98.7) 


None 
None 

6-*7(903) 


8 n 
ZAP 

F=0 


TP 
TP 
TP 


0 


No ID 
3(98.7) 
4(98.7) 


7(98.7) 
None 

1(98.7), 3(98.7), 
6(92S> 7(98.4) 




Ts D 
T« D 




No ID 
6(98.7) 


2(91.4), 7(98.7) 
2(98.7), 7(98.7) 


.ZAP 
r=0 


SP 

SP 

sp 


0 


2(98.7) 
3(98.7) 
8(98.7) 


None 
None 
None 




Transfer Function Cain = 1,50, Learning Rate = 


0.120 


8 • „ 
Z AP 
i=0 


AP.aP,aP 
aP,aP,aP 


13 


0(98.7*1(98.7), 2(98.7) 
3(98.7), 4(98.7), 5(98.7) 
6(98,7), 7(98.7), 8(98.7) 


None 
None 
None 






19 


0(98.7), 1(98.7), 2(98.7) 
3(98.7), 4(No ID), 5(9&7) 
6(98.7), 7(98.7), 8(98.7) 


None 
None 
None 


1 AP 
i=0 


t 0 » 


0 


No ID 


3(98.7), 5(98.7), 
6(98.7) 



t 3 D NoXD 1(97.5), 5(98.7), 
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TABLE 3-continued 



Decimal Data Set Representation 



Training 
Spectrum 


Testing 
Spectrum 


Added 
Noise 
(«) 


Correct ID 
>90% Agent 
(% True) 


Incorrect ID 
>90% 

Agent (% False) 




t o 

V 




No ID 
No ID 
No ID 


8(98.6) 
None 
None 
None 


8 ' r, 




0 








Transfer FiiDCtioa Gain = 1.00, Learn 


ing Rate = 0.200 



Input layer neurons = 350, 1st hidden layer neurons = 256 and 2nd hidden layer neurons = 128, 
output layer neurons = 9, epoch error < 10" 15 , weight limits ±2.5, A D = decimal IR absorption 
spectrum, (Nicolet MX-1 or 10-DX spectrometer measurement), A D = decimal first derivative 
absorption spectrum, i = (0, 1, 2, 3, 4, 5, 6, 7, 8) s (VX, Vx, DIMP, DMMP, GA, GB, GD, GF, 
SP96), 1° = drcinia l Gaussian 90 model theoretical prediction spectrum, t° = decimal first 
derivative theoretical prediction absorption spectrum, S D = decimal interferometer sensor 
spectrum, S B = decimal first derivative interferometer sensor spectrum. 



Given in Table 3 above are network neurons per input- 
hidden-output layers, converged epoch error, and tanh trans- 
fer function parameters including gain and learning rate. The 
summed A- symbol suggests a decimal format of nine scaled 
infrared absorption spectra (page 19) and its first-derivative 
(page 20). With the addition of noise to each spectrum, these 
training data sets were re-submitted as test data sets of the 
trained network filters. Noise is defined as a random per- 
centage of the most intense peak (seed) value in each 
spectrum of the training set; it is added or subtracted 
point-by-point to all training set spectra. Noise above the 
tabulated set; values cause chaos or confusion by the net- 
work to provide accurate target decisions. Testing die trained 
network is also performed with passage of scaled decimal 
data representing theoretical predictions of absorption spec- 
tra and its computed first derivative spectra (T% T°), and 
spectra measured by a TL sensor system (S D t S^). Correct 
and incorrect identification percentages are computed by the 
inner product of training vector and network output vector. 
Results of correct or incorrect identifications use a 90% 
cutoff rule. The dashed entries infer unsuccessful prepara- 
tion (preprocessing) of raw spectra for network submittal 

The network performance results for noisy training data 
between straight absorption and derivative-absorption 
trained networks show resemblance. Network performance 
between binary and decimal based networks cannot be 
compared by virtue of different definitions of noise. In the 
decimal networks, the scaled absorption peak intensity in 
each analyte exemplar was first located. The noise percent- 
age value given in Table 3 is a percentage of this peak value. 
Arandom number generator used this peak value as an upper 
bound per spectrum for distributing extraneous signal to 
each training exemplar (point-by-point random additions 
and subtractions, or no change). In the absorption decimal 
filter, 27% noise in the training data sets cause initial 
network breakdown; here a first false identification of agent 
GF(7) in agent exemplar GD(6) is shown. The derivative- 
absorption decimal filter began showing problems of true 
decision making at 19% noise, where it failed to identify 
agent GA(4). 

Performance by these filters from feed-forwarding deci- 
mal theoretical spectra (T 0 , T^) are next shown in Table 3. 
As with its binary filter counterpart, performance results are 
not good for these decimal filters. Table 3 shows that true 



25 identification with no false positives was made only once 
with submission of the theoretical DMMP absorption deci- 
mal spectrum All other results show true identifications 
shared with false positives, null true decisions, or null true 
decisions with false positives. The conclusion here is the 
same as with the binary filter tests: Theoretical spectra 

30 cannot substitute for experimental spectra when training 
these network filters for successful detection of agents given 
the level of theory used or the methods of Gaussian 92 
computations performed. Better agreement of analyte IR 
absorption frequencies and intensity of absorptions is 

35 required between experiment and theory. The accuracy of 
spectral prediction can be much improved by new and better 
methods of computation. 

Network filtering of the decimal sensor spectra (S D y S°) 
is the next group of results in Table 3. Again, these data lack 

40 resolution and have inherent truncated TL signal in the lower 
spectral band of the FITR instrument The decimal absorp- 
tion network filter results are remarkably accurate and 
reproducible: all true positives and no false positives with 
high confidence. Performance results by the derivative- 

45 based network filter are not shown because of major short- 
comings in preprocessing of these differentiated field sensor 
spectra. As previously noted, derivative data require high 
resolution and special smoothing techniques in preprocess- 
ing. The derivative-absorption based neural filter will be 

50 retested when better sensor data become available. 

FIG. 4 depicts a neural network implemented in hardware 
500, with chip layout and procedure for downloading these 
network architectures and learned weight matrices to the 
Intel IC hardware. Also shown in sensor 501 with Michelson 

55 interferometric and an EFT module. Transferxal to high- 
speed neural chips is necessary for employing these NNCs 
as real-time sensor decision-makers. (Real-time can be bet- 
ter stated as the minimum time of throughput from collec- 
tion of radiance to post-processing of the network output 

60 layer data.) Spectral pattem-niatching and decision making 
tasks are done in short order by design of the 80170NX 
neural chip, firmware housing (EMB board), and the soft- 
ware drivers that control communications in a group of eight 
chips; computational throughputs exceed 16xl0 9 PE inter- 

65 connections per second. (Scan rate of the Michelson inter- 
ferometer mirror is the bottleneck in arriving at a detection 
decision from the NNC system) 
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The modular architecture of FIG. 4 is a manifestation of quality. Performance by the decimal derivative-absorption 

how DynaMind translates the network filters into 80170NX filter from our preprocessing treatments of these sensor data 

IC hardware. All chips 506a . . . 510 are initialized to 12$ is lacking and inaccurate. The problems of poor performance 

input PEs. Chips S06a-S06d cover the rmdinfrarcd spectrum encountered With the derivative-based sensor spectra can be 

in overlapping groups of 128 PEs, 350 independent TL 5 improved with better instrument hardware (new TL stimn- 

spectral amplitudes passed on from the sensor's DSP array lation source, and better instrument resolution) and prepro- 

pre-processing board 502 through input layer 504. The cessing algorithms (better smoothing operation), 

succeeding three neural chips 508c-508c accept output from Training periods by the four-layer network with binary 

the first hidden layer of 256 PEs. Between hidden layers are encoded data are inherently shorter than with a decimal 

eight modules 507 of ue network architecture. Hidden byer 10 format However, in the one of nine problem, network 

2 with chips 50&z-508c produces 128 outputs distributed to feed-forwarding time frames (interval from submission of 

chip 510: here there are 3 network modules 509. The final spectra to the network input layer to output layer result, in 

module is the network output 512 of nine PEs. This final ms) arc similar for all data formats by chip-in-loop hardware 

module represents a nine-dimensional decision vector implementation. The any of N problem can be solved by 

whose nine analog signal components are sent to post- is binary filter using a cascading data procedure. The same 

processing modules 514 that make event and non-event problem with decimal-based filters can be solved by retrain- 

decisions. These no alarm/alarm events are later used in ing the networks with quantitative absorption analyte 

localized coordinate mapping functions, part of the sensor spectra, 2"-l exemplars. The latter network filter would 

post-processing tasks. These mappings include the display yield detection decisions quicker. For implementing the 

of terrain maps, with tracking of alarm icons, routed to 20 decimal filters, complications from adding a bank of D/A 

electronics ground stations using a Global Positioning Sys- converters to the data acquisition system is realized, 

tern. Other functions related to systems automation, signal With these factors in mind, the neural network filter built 

processing, and safe operation of the sensor will be docu- with the binary format structure is best suited for the one of 

mented later. The configuration file required by DynaMind nine 9 TL spectral pattern recognition problem. There is 

for chip transferral of these four-layer binary and decimal 25 room for improvement of network design. False detections 

formatted networks reads thus: by noise in the sensor exceeding those limits established in 



* CFG file for pattern recognition of 9 
agents 

* 8 chip, 4-hyei, back-prop mi to solve the 
♦binary 4th peak detection problem 
*IOHle:a4pkb.» 

* Number of Inputs 

INPUT 350 * 330 inputs over 4 chips: 0-3 

* Number of Outputs 

OUTPUT 9*9 outputs from chip 7 

* From sensor preprocessing 

Layer 1 * Sensor IR spectra] data, 350 PEs 

Extemani40a27]^iipainl0..127] 

Exten»IIi474..2011-auplIn[a.l27] 

ExtemaIIr4148..275K^2Ih[p..l271 

ExtcmalInl222..349Khip3In[0..127] 

Layer 2 * First hidden layer, 256 PEs 

* Next 2 lines split weight matrices 

ChirjOChit[0.31Haup4In[0--311 

Chip0Out[32-63Khip4In[32..63] 

Cm'p0Out[0.31Khip5Ih[0.311 

* Next 2 lines split weight matrices 



CmplOut{0.31>aup4Inl64,S5] 
OiiplOut(32..63]-Chip4Ih[96..127] 

mplOutt0.3lKlirp5In[32.-63] 
Chip2Out[0.31)=€hfc5Itf 64.-95] 

* Next line split into two weight matrices 
Cbip2Ont(0.31]-Chip61n[0„31] 
Cttp20ut[31.te]<;hipcln[32..63] 
Cmp3Ont[0.31>Ch^3In[96..127] 

* Neit line split into two weight matrices 
Orip30ut{p. 3 lKhip6Ifl[64.55] 
Chip30nt[3X.633^hip6hi[96..1271 
Layer 3 • Second hidden layer, 128 PEs 
Chip4Out(0.Jl>Chip7Int0-31] 
C^p5Out[0„63Haup73h£32.,95] 

In[96...127] 
ExtLayer * "any of 9 analytes" space, 9 
PEs 

aup7Out[0 H 8]-ExtsmalOut[0.8] 

* To sensor post-processing 



FIG. 5 shows an overview of the training directed above. 

Binary and decimal based' neural network filters built 
from molecular absorption by nine contaminant compounds 
were presented. These network filters set accurate perfor- 
mance standards by results of tests using spectral data with 
large spurious noise content and/or from spectra with low 
resolution and partly attenuated bandwidth, improved algo- 
rithms for the preprocessing of raw spectral data, in situ, 
especially for the decimal derivative-absorptic, in format of 
difference spectra, are being investigated. 

No strong advantage is seen in filtering performance by 
these networks between submissions of respective binary 
absorption and decimal absorption data formats, with noise, 
in solving the one of nine contaminant problem. 
Furthermore, no advantage in filter performance is shown 
between decimal absorption and its derivative format of 1 
cm -1 resolved laboratory spectra. True and accurate results 
by the straight decimal absorption filter and the binary filter 
were realized from submission of sensor data. These results 
are reproducible, given the sensor data are not of the highest 



Tables 2 and 3 could be costly: network training by the 
gradient descent paradigm does not penalize false positive or 
false negative decisions. For establishing better confidence 
of spectral detection, these analyses can be extended to 
Radial Basis Function (RBF) networks. RBF networks are a 
forward, 3-layer, architecture where summation and activa- 

55 tion function units in the hidden layer combine to form radial 
basis functions. The decision boundary by RBF training is 
rounded in dimensional weight space with radius and center 
point In training, these networks can be made to deactivate 
hidden layer FEs that contribute to a false identification and 

50 make active those hidden layer PEs that arc suppressed in 
positive non-decision events. NNC hardware implementa- 
tion of these IR spectral filters is possible with Intel Cor- 
poration's NilOOO IC chip, a neural network product based 
on the RBF. 

65 Tnernrcluminescence in natural and manufactured mate- 
rials is investigated and applied to a standoff sensor concept 
where: 
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(a) rate of change of liberated TL flux is generated by the first hidden layer has 256 neurons; and 
beam irradiation whose energy is strongly absorbed by the second hidden layer has 128 neurons. 

the surface and coupled into the volume. 8. A system as in claim 7, wherein the output layer has 

(b) A major part of the absorbed energy drives rotational I " nc ncur oas* 

states in water molecules that cause heating of the 5 '.A system as in claim 8, wherein the artificial neural 

neighborhood medium, developing in it a temperature netwo * means comprises: 

gradient between unlike materials. a plurality of artificial neural network chips, each com- 

(c) TL radiance liberated from this temperature non- P ^ a of ** d 

equilibrium zone are collected within the 8-12 mothertoard-bus means for interconnecting the plurality 

micrometer wavelength band by a telescope, 10 of neuraI m : twork chi P s * a serial-parallel 

collimated, passed through aMichelson interferometer, coimguration to provide the input layer, the first and 

then spectrally deconvolved into thermal signature 1n second mdden me out P ut la y er * 

spectra. This is done when the temperature gradient in 10 * A s y stem 35 * claim 1, wherein the sensor means 

the beam irradiation zone is maximum, producing composes* 

maximum contrast of emissivity between target(s) 15 a interferometer for producing raw data; and 

(vibrationally active molecules to be detected) and fast Fouricr transform means for converting the raw data 

background, (bulk matrix) int0 """caring the memoluminescence. 

(d) m this thermal window, contiguous sets of trans- 1L * * ^ artificial neural network to 
formed interf erograms are subtracted, corrected for reC ? gmZe compositions of matter in accordance with ther- 
KlL. ^^Tfi^ a a Ji • .rr* 20 moluminescence exhibited by the compositions of matter, 
base-line drift, scaled, filtered, and cast into binary or me method comprising: 

decimal data formats for , ^ .j. » « * 

, , (a) providing known absorption spectra for the composi- 

(e) pattern recognition by a highly tuned neural network ^ ons 0 f matter; 

rn^ Xei ' i _ j. i i . . - (b) differentiating the known absorption spectra with 

J^I^J^^ 25 rc * cct to abs ^on time to derive duTerentiated 
ernes in the stimulated contaminant within this window of absomtinn soectra- and 
thermal opportunity. If vibrational features arc brought-out „. ,™ _ , , _ ^ ^ , 
in the network output layer, with high confidence, then a (c) training the artificial neural network by backward error 
positive condition is set and electronically routed for post- propagation with the known absorption spectra and the 
processing and advance warning. differentiated absorption spectra to obtain a plurality of 
This disclosure sets forth a preferred embodiment of the 30 neural network weights for the artificial neural net- 
invention. However, those skilled in the art who have W0lk - 

reviewed this disclosure will readily appreciate that other 12. A method as in claim 11, further comprising: 

embodiments within the scope of the invention are possible. (d) providing a second artificial neural network having a 

For example, different software and microprocessor chips same configuration of neurons as the artificial neural 

could be used. Therefore, the invention should be construed 35 network; and 

as limited not by the preferred embodiment, but only by the (e ) exporting the plurality of neural network weights from 

appended claims. ^ ^mctel neural network to the second artificial 

y e A daun: , .. . neural network; 

a reC ° 1 8mZlIlg cwP"^* "ft* k whereby the second artificial neural network becomes 

accordance with thermoluminescence exhibited by the com- 40 £ * . ^ * ^7 

positions of matter, the system comprising: „ c f a We of recognizing toe compositions of matter 

c . ' . .u a i 13. A method as in claim 11, wherein step (c) comprises 

sensor means for detecting the toolummescence; C0DVCltmg ^ laam absorption speclra m £ bin^Talues 

spectral analysis means for analyzing the thermolu- and supplying me binary values to the artificial neural 

minscence detected by the sensor means to produce network. 

spectral data for a plurality of wave numbers; and 45 14. a method as in claim 11, wherein: 

artificial neural network means for receiving the spectral step (c) comprises converting the spectral data into a 

data and for detenriining, in accordance with the spec- plurality of discrete spectral data; and 

tral data, whether the mennoluminescence detected by the artificial neural network comprises one input neuron 

the sensor means indicates the presence of the compo- eacn 0 f foe plurality of: discrete spectral data, 

sitions of matter. 50 15 A metnod ^ ^ claim wher ein the artificial neural 

2. A system as in claim 1, wherein the spectral data are network further comprises one output neuron for each of the 
binary data. compositions of matter to be recognized. 

3. A system as in claim 1, wherein the spectral data are 16. A method as in claim 15, wherein the artificial neural 
decimal data. network means further comprises first and second hidden 

4. A system as in claim 1, wherein: 55 layers. 

the spectral data comprise a plurality of discrete spectral 17. A method as in claim 16, wherein: 

data; and the input layer has 350 neurons; 

the artificial neural network means comprises one input the first hidden layer has 256 neurons; and 

neuron for each of the plurality of discrete spectral data. the second hidden layer has 128 neurons. 

5. A system as in claim 4, wherein the artificial neural 60 18. a method as in claim 17, wherein the output layer has 
network means further comprises one output neuron for each nine neurons. 

of the plurality of compositions of matter to be recognized 19. A method as in claim 11, wherein step (c) comprises 

6. A system as in claim 5, wherein the artificial neural converting the known absorption spectra into decimal values 
network means further comprises first and second hidden and supplying the decimal values to the artificial neural 
lavers - 65 network. 

7. A system as in claim 6, wherein: 

the input layer has 350 neurons; * * * * * 
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